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ABSTRACT 

Td study the effects of practice and incentive on 
learner performance on the aircraft Instrument comprehension task, 48 
third-year Air Force cadets were chosen as subjects. The subjects 
were expected to be able to identify which one of four pictures of 
aircraft in flight most nearly corresponded to the position indicated 
on a panel of attitude and heading instruments. Incentives were in 
the form of additional points and an opportunity to "fly" a formation 
trainer. The practice variable was manipulated by changing the 
instructional materials. The pretest-treatment-post-test experimental 
design provided data for analysis. Data revealed that neither 
variable (practice or incentive) significantly affected the accuracy 
of post-test responses, but they did affect in a positive way the 
post-test rate of responding. The rate of responding is often 
considered to be a good indicator of an individual's productivity. 
Sample instructional materials and a post-test item are appended. 
(AG) 
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PRACTICE AND INCENTIVE EFFECTS ON LEARNER PERFORMANCE: 
AIRCRAFT INSTRUMENT COMPREHENSION TASK 



The present stucly is one of several related research and development 
efforts. The instructional task for these studies is the aircraft instru- 
ment comprehension task. 

The series of studies has been initiated in an attempt to identify 
potentially powerful variables in an effective instructional program. The 
aircraft instrument comprehension program used in these studies was sys- 
tematically designed to include instructional cues, examples, prompted and 
unprompted practice items, feedback, and an incentive. Two tryouts of the 
program were conducted to verify that the students could learn the task 
from the program. The program's effectiveness was established when 80 per- 
cent of the subjects in the second tryout met the instructional objective 
of the program. 

During the two program tryouts, an experiment was conducted to deter- 
mine the effect of feedback on posttest performance. The answers to the 
practice items in the instructional program were available to some subjects, 
but were not available to the other subjects. It was found that the avail- 
ability of feedback in this form did not make a statistically significant 
difference in posttest performance. 

The purpose of the present study was to determine the effects of 
practice and incentive on learner performance on the aircraft instrument 
comprehension task. Practice and incentive were selected from among the 
potentially powerful variables that had been systematically included in 
the program, but that had not yet been manipulated. This study was designed 
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presented. In this study, practice and incentive were added to or deleted 
from the instructional program. 

The incentive variable was manipulated by the use of written instruc- 
tions located at the beginning of the instructional booklet. The instruc- 
tions to the Ss in the "incentive" condition contained the information that 
they could earn up to nine quiz points by responding accurately to the 
items on the posttest. In addition, it was stated in the instructions that 
the four cadets who answered the posttc^st with the greatest accuracy in the 
least amount of time would have an opportunity to "fly" a formation 
trainer. The instructions to the subject, in the "no incentive" condition, 
stated that their grade in the course would not be affected by their parti- 
cipation in the tryout, and that the developers would "appreciate their 
best efforts." 

The practice variable was manipulated by changing the instructional 
materials. In the "practice" versions of the program, two to four practice 
items followed the instruction for each concept. Each item provided prac- 
tice for the instructional objective by eliciting a discrimination between 
two or more positions of aircraft in flight. The multiple-choice practice 
items were similar to the items on the posttest, although some of the 
practice items contained only two response alternatives instead of four 
alternatives. The booklets that included practice contained approximately 
twice as many pages as the booklets that did not include practice items. 
A sample of the practice items, that appeared in the practice versions of 
the program, is contained in Appendix B. 

Instructional booklets were prepared for each combination of the 

practice and incentive variables. One version contained the description 
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of the incentive and the practice iteras. The second version contained the 
description of the incentive, but did not include the practice items. The 
third version contained the practice items, but not the description of tne 
incentive. The fourth version did not include either the description of 
the incentive or the practice items. 

As described above, all four versions of the instructional materials 
contained the pages that explained the concepts. A sample of this instruc- 
tion is contained in Appendix A. The booklets differed by the inclusion or 
exclusion of practice items, and by the inclusion or exclusion of the de- 
scription of the incentive. 

Procedures --A pretest was administered to all S^s one week prior to the 
instruction and the posttesting. The protest consisted of nine items 
selected to be representative of the items on the posttest. A maximum of 
three minutes was allowed to respond to the pretest items. 

Each attended one experimental session in order to read the instruc- 
tional materials and to respond to the posttest items. First, the Ss were 
instructed to complete the instructional booklet assigned to them. Ss 
were asked to record their completion times for the instructional booklets* 
Proctors collected the booklets as the S^s finished them. When all S^s had 
completed the instructional booklets, the posttest was' administered . Ss 
were again asked to record their completion times, and the test booklets 
were collected as the S^s finished. 

Ss were assigned to treatment groups by a procedure that made it 
highly unlikely that any S_s in the "no incentive" condition were aware of 
the incentive. Since the S^s attended class in four different sections, two 
sections were assigned to the "no incentive" condition and all Ss in the 




second two sections were assigned to the "incentive" condition. Within 
each section, half of the Ss were randomly assigned to the "practice" con- 
dition, and the remaining half were assigned to the "no practice" condition 
The mean pretest scores of the four groups were nearly identical. Each of 
the four groups varied from the overall mean score of 6.25 by less than 
one-fourth of one point. 

Criterion Measure- -The posttest contained directions, a sample test 
item as shown in Appendix C, and 36 multiple-choice type test items. Each 
test item consisted of an instrument display containing an attitude indi- 
cator, a heading indicator, and four line drawings of an aircraft in flight 

The test item pool consisted of 72 different positions of an aircraft 
in flight. Three dimensions of the aircraft's position (pitch, roll, and 
heading) were systematically varied to create the item pool. Three levels 
of pitch were used: level, climb, and dive. Three levels of roll were 
used: no bank, 30° right-bank, and 30° left-bank. Eight levels of heading 
were used: four primary compass headings (N, S, E, W) and four intermedi- 
ate compass headings (NE, SE, NW, SW). The posttest items were systemati- 
cally selected to represent equally the variations in heading, pitch, and 
roll. 

Design and Data Analyses- -The pretest-treatment-posttest experimental 
design used in the present study was a 2 x 2 factorial design. Analyses 
of variance were made to determine the effects of practice and incentive on 
(a^) posttest scores, (b^) posttest rates of responding, and {c) instruc- 
tional times. The results of each analysis were tested at the .05 level of 
confidence. 
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Results 

Posttest Scores— The mean scores by treatraent on the 36-item posttest 
are shown in Table 1. The mean scores of the four treatment groups were 
all within one point of the overall mean score of 33.31. The posttest 
scores were analyzed using an analysis of variance. The JF-ratios for in- 
centive, practice, and the interaction of incentive and practice, as shown 
in Table 2, were not statistically significant. 

An increase in the accuracy of performance was observed between the 
pretest scores and the posttest scores. On the pretest, Ss answered a 
mean of 69.4^ of the Items correctly. On the posttest, S^s answered a mean 
of 92.5% of the items correctly. 

Posttest Rate of Responding— Subjects spent differential amounts of 
time answering the items on the posttest. The time for each S.was con- 
verted to a rate of responding by dividing the total number of items by S,'s 
cotnpletion time. The rates of responding were then compared to determine 
whether or not they were affected by the experimental variables. The mean 
rates of responding by treatment are reported in Table 3. The mean rate of 
the incentive group, 5.38 items per minute, exceeded the mean rate of 4.28 
of the no incentive group. The mean rate of 5.42 of the practice group ex- 
ceeded the mean rate of 4.24 of the no practice group. The differences in 
rates of responding were analyzed using an analysis of variance. The re- 
sults of the analysis, as shown in Table 4, revealed that the difference 
attributable to practice was significant at the .001 level of confidence. 
The difference attributable to incentive was also statistically significant 
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TABLE 1 

Mean Posttest Scores by Treatment 
(Number Correct of 36 Possible Points) 





Incenti ve 


Practi ce 






Practice No 


Practi ce 


Totals 


Incenti ve 


33.33 


32.42 


32.88 


Mo Incentive 


33.50 


34.00 


33.75 


Totals 


33.42 


33.21 


33.31 



N = 12 per cell 



TABLE 2 

Analysis of Variance: Posttest Scores 



Source of Variation 


SS 


df 


MS 


F-Ratio 




Incenti ve 


9.19 


1 


9.19 


1 .12 


NS 


Practi ce 


.52 


1 


.52 


.06 


NS 


Incentive X Practice 


6.02 


1 


6.02 


.73 


NS 


Within 


360.58 


44 


8.20 







NS = Not Significant 
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TABLE 3 

Mean Posttest Rates of Responding by Treatment 
(Items Ansv/ered Per Minute) 





Practi ce 




Incenti ve 

Practi ce 




No Practice 


Totals 


Incentive 6.15 




4.61 


5.38 


No Incentive 4.69 




3.88 


4.28 


Totals 5.42 




4.24 


4.83 


N = 12 per cell 


TABLE 


4 




Analysis of Variance: Posttest "Rates 


of Responding 




Source of Variation 


SS 


df MS 


£-Rati 0 


Incenti ve 


14.40 


1 14.40 


22.15* 


Practi ce 


16.56 


1 16.56 


25.48* 


Incentive X Practice 


1 .54 


1 1 .54 


2.37 NS 


Within 


28.70 


44 .65 





* £ < .001 

NS = Not Si gni fi cant 
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at the .001 level of confidence. The interaction of incentive and practice 
was not statistically significant. 

The Ss in the treatment group that read the instruction without either 
practice or the incentive responded at about the same rate on the posttest 
as they had responded on the pretest. Their mean rate of responding was 
3.98 items 'per minute on the pretest. On the posttest, their mean rate of 
responding decreased slightly to 3.88 items per minute. It appears that 
the rate of responding was relatively unaffected by the instructional 
materials. 

Instructional Times — In the comparison of instruction treatments, an- 
other factor worth consideration is the effect of each experimental vari- 
able on instructional time. In the present study, comparisons were made 
to determine whether the practice or the incentive influenced the amount 
of time that Ss spent completing the instructional materials. The mean 
values for instructional times by treatment are shown in Table 5. It can 
be seen from the table that the mean instructional time of 5.11 minutes 
for the incentive group was less than the mean time of 6.63 for the no in- 
centive group. An analysis of variance of instructional times, reported 
in Table 6, revealed that the difference between the time of the incentive 
group and the time of the no incentive group was significant at the .01 
level of confidence. 

Practice, on the other hand, led to an increase in instructional time 
compared to the no practice condition. The practice group had a mean time 
of 8.13 minutes, which is more than twice the mean time of 3.61 of the no 
practice group. An analysis of variance revealed a significant difference 
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TABLE 5 

Mean Instructional Times by Treatment 
(Minutes) 



Incenti ve 


Practi ce 






Practice No 


Practice 


Totals 


Incenti ve 


7.08 


3.15 


5.11 


No Incentive 


9.17 


4.08 


6.63 


Totals 


8.18 


3.61 


5.87 



N = 12 per cell 



TABLE 6 

Analysis of Variance: Instructional Times 















Source of 


Vari ation 


SS 


df 


MA 


f- Ratio 


Incenti ve 




27.38 


1 


27.38 


11 .04* 


Practi ce 




244.13 


1 


244.13 


98.44** 


Incentive X 


Practi ce 


3.94 


1 


3.94 


1.59 NS 


Within 




109.06 


44 


2.48 





* £ < .01 
** £ < .001 

o 
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at the .001 level of confidence between the instructional time of the prac- 
tice group and the time of the no practice group. 

Discussion 

The present study was conducted to determine the effects of practice 
and incentive on learner posttest performance on an aircraft instrument 
comprehension task. The results of the study did not indicate that either 
variable significantly affected the accuracy of responding on the posttest. 
Ss in the treatment group that simply read the instructional materials had 
a mean score of 34 on the 36-item posttest. Such a high mean score left 
little room for improvement that could be attributed to either the practice 
or the incentive variable. The increase in accuracy that was observed be- 
tween the pretest and the posttest appears to be attributable to the 
instructional materials. 

Although no significant differences in accuracy of responding resulted 
from the manipulation of the experimental variables, differences attribu- 
table to practice and to incentive were observed in the posttest rate of 
responding. The rate of responding was significantly increased both by the 
incentive that was offered and by the practice that was provided. 

The rate of responding is an important aspect of the instrument com- 
prehension task in that it indicates the rate at which pilots read instru- 
ments accurately and quickly is a highly desirable skill to acquire in 
pilot training. 

The rate of responding is often considered to be a good indicator of 
an individual's productivity. Judgments about individuals in work situa- 
tions are often based on the amount of work that an individual produces, as 
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well as, the quality of the work. Tests that measure the rate of respond- 
ing, such as tests for typing and reading, are used to predict the quantity 
of work that an individual can produce. Quantitative measures are especi- 
ally important in the comparison of individuals whose work is qualitatively 
comparable. The results of this study indicate that practice and incentive 
could potentially affect an individual's productivity by increasing the 
rate at which he can perform a task. 

A difference in the rate of responding attributable to incentive was 
observed in the present study. The incentive was contingent upon both the 
speed and the accuracy of Ss' responses. In the "no incentive" condition, 
however, neither speed nor accuracy was reinforced. It is likely that S^s 
in the incentive condition were attempting to respond as fast as they 
could without making errors. S_s in the "no incentive" group, however, 
were probably trying to answer correctly without attempting to answer 
rapidly. The incentive, therefore, most likely served to increase the S_s' 
efforts on the posttest rather than to increase their ability to respond 
rapidly. On the other hand, the shorter instructional time of the incen- 
tive group indicates that these S^s may have practiced fast responding 
during the instruction. This practice could have increased their ability, 
not just their efforts, to answer rapidly on the posttest. 

The increased rate of responding attributable to practice in the pre- 
sent study was achieved at the expense of an increase in instructional 
time, since the booklets containing practice were longer than the booklets 
without practice. Whether or not the additional instructional time is 
justified by the increased rate of responding is a matter of judgment. In 
making such judgments, consideration should be given to the nature of the 
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task. For example, if one's safety depended on a short reaction time, as 
in an c^ircraft navigation task, the value of the rate of responding would 
be different than if one's safety depended pnly on an accurate response. 

Recommendations for Future Studies— The series of studies to determine 
the variables that are contributing to the effectiveness of the aircraft 
instrument comprehension program indicates that^ feedback, practice and in- 
centive have not significantly affected the accuracy of learner responses 
on the posttest. It appears that the instructiqpal materials are making 
the major contribution to the acciiracy of learr\ex responses. Instruction 
could be manipulated in a fixture study in order to verify its contribution 
to posttest responses. 

The results of the present study indicate that practice and incentive 
may produce statistically significant increases in the rates of responding, 
but not in the accuracy of responding, when they are used in combination 
with an effective instructional program. Since practice and incentive are 
generally considered to facilitate learner achievement, it is hypothesized 
that the findings of the present study are generalizable only to learning 
situations that use instructional materials that are effective in attaining 
the instructional objectives. One method of testing this hypothesis might 
be to administer the aircraft instrument comprehension program to learners 
who score low on the pretest. If the program is less effective with these 
learners, then practice and incentive might make a significant contrih'ition 
to learner performance. The generalizabili ty of the present findings could 
also be tested by varying practice and incentive with different instruc- 
tional materials that are less effective than the materials used in the 

present study. Future research about the conditions under which practice 
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and incentives facilitate learning should enable instructional designers 
to more effectively incorporate these two variables into instructional 
programs. 

The follflifing supporting documents are in the R & D Case File at 
Flying Training Division, Air Force Human Resources Laboratory, Williams 
Air Force Base, Arizona 85224: 

Higgins, N. C. Aircraft instruwent con}rehension program; Fo rm B. (AFSC 
Contract Ho. F41 609-71 -C-0027, Task Order Ho. 3) Teime, Arizona: 
Arizona State University, 1973. 

Higgins, H. C, A Kearns, D. R. Validation report: Aircraft instrument 
comprehension: A self-ins triictional program: Fom A . <AFSC nnntrart 
No. F41609-71-C-0027 Task Order Ho. 3) Tenipe, Arizona: Arizona State 
University, 1973. 

Higgins, H. C, Keams, D. R., & Tenpas, B. G. Validation report: Air- 
craft instrument comprehension: A self-instructional orogra m: 
FonnJI. (AFSC Contract Ho. E41 609- 71 -C-0027, Task Order Ho. 3) Tenpe 
Arizona: Arizona State University, 1974. 

Kearns, D. R., Tenpas. B. G., & Higgins, N. C. Aircraft instrument comp re- 
hension test: Form B . (AFSC Contract No. F41609.-71-C-0027, Task 
Order No. 3) Tempe, Arizona: Arizona State University, 1973. 
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Attitude Indicator-Pitch 



The instrument labeled attitude 
airplane is climbing or diving, 
the degree of bank to the right 

The small aircraft silhouette in 
stationary. The position of the 
the horizon, varies with the air 



indicator shows whether the 
This instrument also shows 
or left. 

this instrument remains 
heavy black line, representing 
plane's position. 





When the airplane is 
flying level, the 
horiion line wi 11 be 
directly on the air- 
craft silhouette as 
shown above. 




ATTirUD!! ' 
lfJD!CAT0;7 , 



If the airplane is 
climbing, the silhouette 
is seen betv/een the 
horizon line and the 
triangular pointer. 




ATTITUDH 
IN'DICATC^ 



If the airplane is 
diving, the horizon 
line will be between 
the silhouette and 
the pointer. 
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The airplane in Figure A is 
climbing on a northwest heading. 



The horizon line is below 
the aircraft silhouette in 
the attitude indicator. The 
arrow in the heading indicator 
is pointing midway between the 
N and the W. 





ATTrfUO:: 




ERIC 



The airplane in Figure B is 
(living on a souiheost noadino. 



The horizon line is above the 
aircraft silhouette in the 
attitude indicator. The 
arrow in the heading in- 
dicator is pointing mid- 
way between the S and the 
E. .21 
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Which of the airplatniS pictured helov/ s most nearly in the 
pObiLion ihilicaLeil on I'lc oUUmio iinJ heading indicators? 





Which of the aircraft pictured below is in the position 
shown on the instrument panel? 

Mark your choice opposite item number 4 on your 
response sheet. 
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Sample Posttest Item 




